Method and apparatus for remapping pixel locations

ABSTRACT

An apparatus and method for optically remapping projected pixels to maximize the utilization and to optimize the distribution of remapped projection pixels to achieve optimal visual performance (generally uniform resolution and luminance). A device interposed between a projector and an imaging surface for optically remapping projected pixel locations with minimal aberration. When this device is interposed between a projector and an imaging surface, it changes the terminal location of each focused pixel such that it maximally coincides with the imaging surface, which is often a surface of complex curvature and very different from the native focal surface of the projector. One implementation of the technology includes a device that uses multiple optical surfaces.

CROSS-REFERENCE

This application is a Continuation application of, and claims thebenefit of priority to, U.S. patent application Ser. No. 17/474,838,filed Sep. 14, 2021, which issued as U.S. Pat. No. 11,595,626, whichclaims the benefit of and priority to U.S. patent application Ser. No.16/688,795, filed Nov. 19, 2019, which issued as U.S. Pat. No.11,122,243, which claims the benefit of and priority to U.S. ProvisionalPatent Application Ser. No. 62/769,368, filed Nov. 19, 2018, andentitled DEVICE FOR OPTICALLY REMAPPING PROJECTED PIXEL LOCATIONS WITHMINIMAL ABERRATION, which are both hereby incorporated herein in theirentirety.

FIELD

This technology as disclosed herein relates generally to opticalremapping of projected pixel locations and, more particularly, toremapping with minimal aberration.

BACKGROUND

The cost and performance efficiency of a projector-based visual displaysystem is primarily dependent upon the utilization of availableprojector pixels. Commercially available projectors are designed toprovide a rectangular projection frustum, with square pixels, andnegligible field curvature. A visual display system is generallydesigned to achieve a required Field Of View (FOV), resolution, andluminance. Since the most efficient distribution of pixels for humanobservation is that of a visual sphere, and a rectangle does nottessellate onto a sphere with high efficiency, these visual requirementsgenerally result in poor utilization of the rectangular, square-pixel,projection frustum.

There are special lens systems, namely fish-eye (“f-theta”) andanamorphic lenses which enable some adjustment of the rectangularprojection frustum, but they are inadequate. Fish-eye lenses havelimited tolerance for the location of the origin of projected light,which can force the projector to be located within the viewing area orother physical obstruction. Further, Fish-eye lenses also have lesstolerance for tuning the local distribution of pixels. An afocalanamorphotic cylindrical lens system which is set up in the projectionpath of rays of a spherical projection and camera objective lens on theside of the longer distance between back lens and image and serves forobtaining an image sharp in all image points. Anamorphosing cylindricallens systems have become known which consist of a lens componentpositioned in front of the objective having a positive cylindrical powerof refraction, and another lens component, separated from the first byair and having a negative cylindrical power of refraction the cylinderaxes of these members being parallel to each other with each of said twocomponents formed of a converging lens cemented to a diverging lens andwith the cylindrical cemented surface in the diverging component havinga converging effect and with its concave surface turned toward theconverging component, and the cemented surface of the convergingcomponent having a diverging effect and with its converging surfaceturned toward the diverging component. Axes of the two components lie invertical planes so that the system in the horizontal plane decreases thefocal length of the objective while in the vertical plane the focallength remains unchanged, i.e. in the horizontal plane a change of theimage scale is affected while in the vertical plane the image scaleremains unchanged. However, anamorphic lenses are also relativelyintolerant of the tuning of the local distribution of pixels.Commercially available anamorphic lenses are also limited to smallchanges in aspect ratio.

Another objective having become known up to now consists of at leastfive lenses the diverging components being made up of three lenses. Inspite of the number of lenses and the arrangement of these known systemsthe projection quality, especially in respect to coma and distortion, isnot satisfactory.

A better apparatus and/or method is needed for improving opticalremapping of projected pixel locations and, more particularly, toremapping with minimal aberration.

SUMMARY

The technology as disclosed herein includes a method and apparatus foroptically remapping projected pixels to maximize the utilization and tooptimize the distribution of remapped projection pixels to achieveoptimal visual performance (generally uniform resolution and luminance).By way of illustration, fish-eye lenses are utilized in an attempt tomeet this need, but Fish-eye lenses have less tolerance for tuning thelocal distribution of pixels, whereas a slightly flexible mirror element(thin glass, e.g.) as proposed herein for the present technology and itsvarious embodiments and implementation allows for such adjustmentwithout remanufacture of the optical element.

One implementation of the technology is a device interposed between aprojector and an imaging surface for optically remapping projected pixellocations with minimal aberration. When this device is interposedbetween a projector and an imaging surface, it changes the terminallocation of each focused pixel such that it maximally coincides with theimaging surface, which is often a surface of complex curvature and verydifferent from the native focal surface of the projector. Oneimplementation of the technology includes a device that uses multipleoptical surfaces. However, one implementation of technology includes asingle optical surface that is effective when the remapping-inducedaberrations (focus blur) are less than the required resolution for thevisual display system. Yet another implementation of the technologyincludes a device that uses a combination of reflective and refractiveoptical surfaces, however, visual display system performance is improvedby a purely reflective or refractive set of optical surfaces. For oneimplementation of the invention one optical element is utilized to bendthe light and the other is used to correct the focus.

One implementation of the technology includes a projector that providesa projected light frustum incident onto a largely cylindrical refractivelens. The center of curvature of this lens is proximately coincidentwith the light origin point of the projector. This element enablescorrection of anisotropic focus, which is a product of the degree ofangular deviation effected by downstream elements in order to achievethe remapping. It does so because the chief ray (central ray) from thelight origin is largely normal to the refracting optical surface and soexperiences minimal deviation, but the marginal rays (from edges of thedefocused spot at light origin) experience significant deviation, whichchanges the focal depth—but not the location—of a given pixel. For oneimplementation of the technology as disclosed and claimed herein thislens uses multiple elements to reduce chromatic dispersion (achromaticdoublet, e.g.). Light exiting the aforementioned lens is then incidentupon one or more reflective optical surfaces which remap the projectedpixel locations by way of non-planar surface geometry.

For one implementation of the technology as disclosed and claimed hereina single reflective surface is used, and the reflective surface isprovided by a first-surface-coated thin glass substrate (for oneimplementation about approximately 200 micron thickness, e.g.). Thisthin mirror is forced into a nearly-cylindrical hyperbolic paraboloid.The thin glass mirror provides significant anisotropic magnificationproducing an elongated aspect ratio when the pixels are finally incidentupon the imaging surface. It also generates significant field curvaturein the same axis. In “cross-cockpit collimated” visual display systemsas well as “dome” visual display systems it is common to employ aspherical, toroidal or other quadric imaging surface, which is muchwider than it is tall. The aforementioned thin glass mirror adjusts thepixel locations and focal surface to more optimally converge on saidimaging surface. This enables a single higher-resolution projector, toreplace two or more lower-resolution projectors. The thin mirrormaterial can be constructed of any material that can be polished to therequired optical smoothness and/or on which a reflective coating isapplied. The mirror has a reflective coating or a reflective portion,and has another portion that holds the structure itself (the substrate).For one implementation the substrate is constructed of one or more of,glass, plastic, metal, metal composite, acrylic, ceramic and carbonfiber. One implementation of a reflective coating includes one or moreof aluminum, silver or a comparable material on glass. The reflectivecoating can be applied to a plastic film, rigid plastic or acrylic.Various manufacturing processes for the substrate and the reflectivecoating can be used without departing from the scope of the invention,

The surface geometry of the thin glass mirror—by way of mechanicalpressure—can be tuned to achieve a free-form optical shape, which moreoptimally distributes pixels per visual display system requirements.This enables accommodation for deviations from nominal in the surfacegeometry of the screen, or deviations in the mapping of the FOV anglesonto the imaging surface. For one implementation of the technology, aproprietary software analysis package (MATLAB code & GUI) is utilized,which enables the empirical (human-in-the-loop) determination offree-form and aspheric geometries to optimally remap pixels via themechanical tuning of the mirror.

Visual display systems for regulatory-certified aviation simulationtraining use multiple projectors with overlapping images to providecertifiable visual performance. The technology as disclosed and claimedherein provides the ability to replace these projectors with a singleprojector. This dramatically reduces the cost and complexity ofmaintaining the ‘ alignment’ (the color, luminance, gamma, and geometricperformance) of adjacent channels, and eliminates the need for costly,error-prone ‘blending’ of adjacent channels. Poor utilization ofprojector pixels costs the Audio-Visual (A/V) and Visual Simulation(VizSim) markets millions of dollars annually. If pixels can beoptically remapped, fewer projectors are required, and there is lessneed for multi-projector alignment systems. Luminance, dynamic range,and system are all increased.

For one implementation of the technology, a visual system design tool isutilized, which could then be modified to simulate the lens behavior forthe proposed system. For the technology as disclosed and claimed herein,the technology as disclosed herein provides for a more advanced controlof pixel location via the mirror shape, and produces curvilinear andlocal pixel redistribution rather than just an angularly uniformstretch. In particular, visual display system applications generallyneed a combination of anamorph and local pixel redistribution. Were aFisheye Lens used, significant pixels would be lost. For oneimplementation of the technology, solutions can be tuned for variousimaging surface geometries. Focus aberrations become quickly apparent asdegree of remapping increases. The technology as disclosed and claimedherein seeks to address the focus issue. One implementation of thetechnology as disclosed and claimed is a device to remap projectorpixels with substantially maximal pixel utilization and substantiallyoptimal resolution for an observer or observers of a display system. Thedevice is interposed within the optical path between image formationwithin a projector and later image formation upon a curved screen. Thedevice includes at least a refractive element and a reflective elementwhich are positioned in optical subsequency. The focal surfaces of therefractive and reflective elements each have a longest dimension, andsaid focal surface longest dimensions are oriented substantiallyorthogonal to one another. For one implementation, the device first usesrefraction to substantially astigmatize the projected image's focus to adegree, which is substantially inverse to the focal astigmatismintroduced by optically subsequent elements. For one implementation, thedevice next uses reflection to supplementarily optically redirect light,a.k.a. remap pixels, from a projector to substantially optimal locationson a curved screen. The device is adjustable in its optical effect inorder to support any of the following: various projectors, variousprojector configurations, various display systems, various displaysystem configurations, variations in display system components,variation in observer location variation in required field of viewand/or resolution. Fewer projectors are required, and less need formulti-projector alignment systems. Luminance, dynamic range, and systemMTBF are all increased. A device can be tunable if needed. The thinmirror configuration having thin glass minimizes chromatic aberrationand enables large magnification/large remapping. The features,functions, and advantages that have been discussed can be achievedindependently in various implementations or may be combined in yet otherimplementations further details of which can be seen with reference tothe following description and drawings.

The technology as disclosed and claimed herein is particularlyapplicable for applications for larger remapping with higher pixeldensities. Higher pixel densities requiring larger remappings havenecessitated a newer solution and is one of the reasons for the presenttechnology as described. Further, the present technology provides forcorrecting resolution from a DEP. The prior art does not do largeremapping of high pixel densities to a remapped location that would beoptimal from a DEP with uniform resolution across the field of view.Further the present technology is tunable into a freeform which is nottaught by the prior art. By way of illustration, for collimated systems,one implementation of the technology provides an eye point for at leasttwo observers so the technology as disclosed and claimed optimizes thesystem for the best balance between multiple observers so when the termdesign eye position (DEP) is utilized herein, for one implementation, itis a DEP that is a balance between the eye positions of multipleobservers. The DEP in this case represents the position of manyobservers, therefore when the term observer is utilized herein it canrefer to multiple observers. The adjustment parameters of the system canbe adjusted with respect to an observer's position or observers'positions. These and other advantageous features of the presenttechnology as disclosed will be in part apparent and in part pointed outherein below.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the present technology as disclosed,reference may be made to the accompanying drawings in which:

FIGS. 1A and 1B are illustrations of an imaging surface of a simulator;

FIGS. 1C and 1D are illustrations of a thin mirror interposed between aprojector and an imaging surface and the design eye point;

FIG. 2 is an illustration of the mapping of pixels to an imagingsurface;

FIGS. 3A through 3C are illustrations of an application in a“cross-cockpit collimated” visual display systems or “dome” visualdisplay system;

FIGS. 4A-4B are illustrations of a set of viewing vectors generated fromthe DEP to represent the projection of the field of view;

FIG. 5 is a graphical illustration of a freeform; and

FIG. 6 is a block diagram illustrating a method of the currentdisclosure.

While the technology as disclosed is susceptible to variousmodifications and alternative forms, specific implementations thereofare shown by way of example in the drawings and will herein be describedin detail. It should be understood, however, that the drawings anddetailed description presented herein are not intended to limit thedisclosure to the particular implementations as disclosed, but on thecontrary, the intention is to cover all modifications, equivalents, andalternatives falling within the scope of the present technology asdisclosed and as defined by the appended claims.

DESCRIPTION

According to the implementation(s) of the present technology asdisclosed, various views are illustrated in FIGS. 1-6 and like referencenumerals are being used consistently throughout to refer to like andcorresponding parts of the technology for all of the various views andfigures of the drawing. Also, please note that the first digit(s) of thereference number for a given item or part of the technology maycorrespond to the figure number in which the item or part is firstidentified. Reference in the specification to “one embodiment” or “anembodiment”; “one implementation” or “an implementation” means that aparticular feature, structure, or characteristic described in connectionwith the embodiment or implementation is included in at least oneembodiment or implementation of the invention. The appearances of thephrase “in one embodiment” or “in one implementation” in various placesin the specification are not necessarily all referring to the sameembodiment or the same implementation, nor are separate or alternativeembodiments or implementations mutually exclusive of other embodimentsor implementations.

The technology as disclosed and claimed herein is a method and apparatusfor optically remapping projected pixels to maximize the utilization andto optimize the distribution of remapped projection pixels to achieveoptimal visual performance (generally uniform resolution and luminance).One implementation of the technology is a device interposed between aprojector and an imaging surface for optically remapping projected pixellocations with minimal aberration. When this device is interposedbetween a projector and an imaging surface, it changes the terminallocation of each focused pixel such that it maximally coincides with theimaging surface, which is often a surface of complex curvature and verydifferent from the native focal surface of the projector. Oneimplementation of the technology includes a device that uses multipleoptical surfaces. However, one implementation of technology includes asingle optical surface that is effective when the remapping-inducedaberrations (focus blur) are less than the required resolution for thevisual display system. Yet another implementation of the technologyincludes a device that uses a combination of reflective and refractiveoptical surfaces, however, visual display system performance is improvedby a purely reflective or refractive set of optical surfaces. Thetechnology as disclosed and claimed herein is particularly applicablefor applications for larger remapping with higher pixel densities.Higher pixel densities requiring larger remappings have necessitated anewer solution and is one of the reasons for the present technology asdescribed. Further, the present technology provides for correctingresolution from a DEP. The prior art does not do large remapping of highpixel densities to a remapped location that would be optimal from a DEPwith uniform resolution across the field of view. Further the presenttechnology is tunable into a freeform which is not taught by the priorart.

Referring to FIGS. 1A and 1B an illustration of an imaging surface of asimulator is provided. FIG. 1A illustrates the surface on which theimage is projected and FIG. 1B illustrates an image being projected onthe surface 101. The imaging surface is shown as a surface of complexcurvature and very different from the native focal surface of theprojector, which teaches an apparatus and method as disclosed andclaimed herein for optimizing the distribution of remapped projectionpixels to achieve optimal visual performance (generally uniformresolution and luminance). A visual display system such as a flightsimulator as illustrated in FIG. 1B is one application of the technologyas disclosed and claimed herein.

Referring to FIGS. 1C and 1D, an illustration of a thin mirror 102interposed between a projector 104 and an imaging surface 106 and thedesign eye point 108 is illustrated. One implementation for providing athin film Mylar mirror 102 is illustrated in FIG. 1D, where the thinfilm Mylar mirror is formed over a base in order to set the shape and atensioning frame is used to form the shape of the base into the Mylarfilm.

Referring to FIG. 2 , an illustration of the mapping of pixels to animaging surface is illustrated. The on screen field of view observedfrom the Design Eye Point and pixel mapping are illustrated. Referringto FIGS. 3A-3C, an illustration of an application in a “cross-cockpitcollimated” visual display systems or “dome” visual display system. Thelight leaves the projector, passes through the refractive element, thenreflects off of the reflective element, then is incident upon thescreen.

Referring to FIG. 4 an illustration of a set of viewing vectorsgenerated from the DEP to represent the projection of the field of viewis shown. Referring to FIG. 5 a graphical illustration of a freeform isprovided. This type of mirror will be referred to herein as a“free-form” fold mirror. The free-form fold mirror can be shaped toreduce or remove the loss of resolution of image components near theboundaries of the projected light cone. Freeform Optical surfaces aredefined as any non-rotationally symmetric surface or a symmetric surfacethat is rotated about any axis that is not its axis of symmetry. Thesesurfaces can lead to a smaller system size as compared to rotationallysymmetric surfaces. The free-form mirror when illuminated by a pointlight source produces a given illumination pattern on a target surfacethat could be flat, spherical or of other shape. For one method ofaccomplishing the design, the optical ray mapping can be modeled bysecond order partial differential equations. For another method foraccomplishing the design, an approximation of the optical surfaces canbe modeled and validated through ray tracing and design of the opticalsurfaces, particularly the free form mirror can be determined. Inaddition to the first curved portion, the free-form mirror can include asecond curved portion spaced from the first curved portion. In thisarrangement, the first flat portion is between the first curved portionand the second curved portion. Additional curved portions can be addedas desired or necessary.

One objective Free Form Mirror technology is to equalize the size andspacing of projector pixels to create uniform resolution so theappearance of the image is consistently sharp. Any geometriccorrections, such as pre-distorting a square to have a “barrel” shape sothat it looks square to the observer instead of having the corners lookelongated, will be done by the Image Generator creating the image, andnot by the mirror. Prior technologies have not addressed uniformresolution. Another objective of the technology is to provide a foldmirror that creates a uniform pixel density (therefore uniformresolution & more uniform brightness) on any screen surface. Thefree-form fold mirror 102 described herein would adjust the distributionof the projector's light rays onto the screen 106 to equalize theresolution across the image and produce a much more uniform resolutionand brightness. By way of illustration, for one implementation ofdesigning manufacturing and providing for a freeform mirror, a ray traceoptimizer that includes a computer based software tool for modeling theray traces and ultimately the free form shape of the fold mirror isutilized. The freeform mirror could be premanufactured utilizing such acomputer based tool. However, for one implementation, the free formmirror is dynamically adjusted with a mechanical push/pull system thatmechanically deforms the reflective surface of the mirror to theappropriate curvature. For one implementation, the push/pull mechanismis computer controlled to adjust the curvature of the mirror based onother system parameters in order to reduce aberrations and improveresolution.

The details of the technology as disclosed and various implementationscan be better understood by referring to the figures of the drawing.Referring to FIGS. 1-5 , one implementation of the present technology asdisclosed and claimed comprises a device 102 interposed between aprojector 104 and an imaging surface 106 for optically remappingprojected pixel locations with minimal aberration, whereby, when thisdevice is interposed between a projector and an imaging surface, itchanges the terminal location of each focused pixel (See FIG. 2 forillustration) such that it maximally coincides with the imaging surface,which is often a surface of complex curvature and very different fromthe native focal surface of the projector, which teaches an apparatusand method as disclosed and claimed herein for optimizing thedistribution of remapped projection pixels to achieve optimal visualperformance (generally uniform resolution and luminance).

One implementation of the technology as disclosed and claimed herein isa method including determining the display system observer location,a.k.a. Design Eye Point 108, along with the required field of view 110,and required resolution for each portion of field of view. For oneimplementation the method includes generating a set of viewing vectors402 generated from the DEP 108 to represent the projection of the fieldof view 110 onto the image formation surface. The viewing vectors 402are intersected with a curved screen whereupon the image is formed. Foreach portion of the field of view, the screen intersection locationsdetermine the location to which projector pixels will be substantiallymapped. The required resolution for a given portion of field of viewdetermines the optimal density of pixels within said portion of field ofview.

One implementation of the method includes interposing a device 102within the optical path 404 between image formation within a projectorand the later image formation 101 upon a curved screen. For oneimplementation, the device includes at least a refractive element and areflective element which are positioned in an optically subsequentorder. The focal surfaces of the refractive and reflective elements eachhave a longest dimension, and said focal surface longest dimensions areoriented substantially orthogonal to one another. For oneimplementation, the method includes adjusting the adjustable device'soptical effect in order to support any of the following: variousprojectors, various projector configurations, various display systems,various display system configurations, variations in display systemcomponents, variation in observer location variation in required fieldof view and/or resolution. One implementation of the methodsubstantially astigmatizes the projected image's focus by way ofrefraction and then uses reflection to supplementally optically redirectlight from a projector to said screen intersection locations with saidoptimal pixel density. The method includes optimizing the utilization ofthe projector's pixels with the device by satisfying the requiredresolution for the given field of view with a minimum of projectorpixels.

One implementation of the technology as disclosed and claimed hereinincludes a device to remap projector pixels (See item 202 forillustration) with maximal pixel utilization and optimal resolution foran observer or observers of a display system. The device is interposedwithin the optical path between image formation within a projector andlater image formation upon a curved screen. For one implementation ofthe technology, the device includes at least a refractive element and areflective element which are positioned in optical subsequency. Thefocal surfaces of the refractive and reflective elements each have alongest dimension, and said focal surface longest dimensions areoriented substantially orthogonal to one another. The device isconfigured to use refraction to substantially astigmatize the projectedimage's focus to a degree which is substantially inverse to the focalastigmatism introduced by optically subsequent elements. For oneimplementation the device is further configured to use reflection tosupplementally optically redirect light, a.k.a. remap pixels, from aprojector to optimal locations on a curved screen. For oneimplementation, the device is configured to be adjustable in its opticaleffect in order to support any of the following: various projectors,various projector configurations, various display systems, variousdisplay system configurations, variations in display system components,variation in observer location variation in required field of viewand/or resolution.

For yet another implementation, by way of application when used as a“cross-cockpit collimated” visual display systems or “dome” visualdisplay system, the curved screen is viewed through a collimating mirror(See item 302 for illustration), whereby the required total horizontalfield of view exceeds 180 degrees, and the resolution is less than 6.1arcminutes per optical line pair, and at least the reflective element isadjustable. For on implementation substantially orthogonal is between 70degrees and 110 degrees.

For the projection system, the image is formed within the projector andprojected for being formed on the screen. For one implementation, theprojector may include a projection lens. In the case of two or moreobservers, a compromise design eye position (DEP) is determined uniquelyfor each portion of the field of view.

For one application of the technology, a curved screen is viewed througha collimating mirror, in which case, the viewing vectors reflect off ofthe collimating mirror and intersects at the intersection with thecurved screen. For one implementation, the curved screen is a section ofan ellipsoid or toroid, or minor variation therefrom. Said minorvariation does not exceed about approximately 0.25 times the largestcurved screen radius of curvature. For one implementation, the curvedscreen includes one radius of curvature which is approximately infinite.

Pixel density refers to the number of pixels per unit screen area. Theprojector's resolution may exceed its pixel count by way of pixelshifting. For one implementation, the refractive element is positionedalong the optical path, before, after or within the projection lens. Forone implementation, planar fold mirrors may be employed throughout theoptical path, about or between any optical elements; from imageformation within the projector to image observation at the design eyeposition (DEP).

For one implementation, the refractive elements incorporate multiplesub-elements and multiple refractive optical surfaces. For oneimplementation, the refractive element has one or more optical surfacessubstantially having the shape of a generalized cylinder with a focalsurface substantially near, or encompassing, the location of the vertexof the projector's projected light frustum. For one implementation, therefractive element has one or more optical surfaces substantially havingthe shape of a freeform with a focal surface substantially near, orencompassing, the location of the vertex of the projector's projectedlight frustum. (freeform defined per ISO standard 17450-1:2011). For oneimplementation, the refractive element has one or more optical surfacessubstantially having a Gaussian curvature of approximately zero with afocal surface substantially near, or encompassing, the location of thevertex of the projector's projected light frustum.

For one implementation, the refractive element is adjustable in itsoptical effect, such that the refractive element is adjustable to afreeform shape (freeform defined per ISO standard 17450-1:2011). For oneimplementation, the refractive element adjustment is achieved bymechanical and/or thermal deformation of any of the refractive surfaces.For one implementation, the refractive element adjustment is achieved byadjustment of relative locations of sub-elements.

The reflective element is positioned along the optical path, after therefractive element and the projection lens. For one implementation thereflective element may incorporate multiple sub-elements and reflectiveoptical surfaces. For one implementation the reflective elementsubstantially has the shape of a generalized cylinder. For oneimplementation the reflective element substantially has a freeform shape(freeform defined per ISO standard 17450-1:2011). The reflective elementsurface substantially has a Gaussian curvature of approximately zero.

For one implementation, the reflective element is adjustable in itsoptical effect. The reflective element is adjustable to a substantiallyfreeform shape (freeform defined per ISO standard 17450-1:2011). Thereflective element adjustment is achieved by mechanical and/or thermaldeformation of the reflective surface. For one implementation, thereflective element adjustment is achieved by adjustment of its locationrelative to the projector, and/or its location relative to therefractive element, and/or the relative locations of the reflectiveelements sub-elements. The refractive element is close enough to thereflective element that the projected light traverses the refractiveelement before and after reflection from the reflective element.

The various implementations and examples shown above illustrate a methodand system for a software analysis package (MATLAB code & GUI) that isutilized, which enables the empirical (human-in-the-loop) determinationof free-form and aspheric geometries to optimally remap pixels via amechanical tuning of the mirror.

Referring now to FIG. 6 , a block diagram illustrating a method 600 foroptically remapping projected pixel locations is generally illustrated.Operation 602 includes defining a design eye point (DEP), a field ofview corresponding to the DEP, and an image resolution for a pluralityof portions of the field of view when viewing a projected image. Inoperation 604, a set of viewing vectors from the DEP representative of aprojection of the field of view onto an image formation surface aregenerated. The viewing vectors intersect a curved screen whereupon theprojected image will be formed. Operation 606 comprises determining, foreach of the plurality of portions of the field of view, screenintersection locations to which projector pixels will be substantiallymapped. An optimal density of pixels within each of the plurality ofportions of field of view based on the image resolution for a givenportion of the field of view is determined in operation 608.

Operation 610 includes interposing an optical device within an opticalpath between an image source formation point within a projector and aprojected image formation point upon the curved screen. The projectedimage is refracted and reflected with the optical device in operation612. The optical device may include at least a refractive element and areflective element positioned in optical subsequence. The focal surfacesof the refractive and reflective elements each have a longest dimension.The longest dimensions of the focal surfaces are oriented substantiallyorthogonal one with respect to the other.

Operation 614 includes adjusting the optical effect of the opticaldevice in order to support one or more of various projectors, variousprojector configurations, various display systems, various displaysystem configurations, variations in display system components,variation in observer location, variation in field of view, andvariation in image resolution. In one embodiment, adjusting the opticaleffect of the optical device includes one or more of mechanicaldeformation and thermal deformation of any of the refractive opticalsurfaces of the refractive element. Adjusting the optical effect of theoptical device may optionally include adjustment of relative locationsof sub-elements of the refractive element. Additionally, oralternatively, adjusting the optical effect of the optical deviceincludes one or more of mechanical deformation and thermal deformationof a reflective surface of the reflective element. Operation 614 mayalso include adjustment of one or more of: a location of the reflectiveelement relative to the projector; a location of the reflective elementrelative to the refractive element and the relative locations ofsub-elements of the reflective element.

The projected image's focus is astigmatized in operation 616 by way ofrefracting with the refractive element of the optical device andreflecting the astigmatized refracted image with the reflective elementa the optical device to supplementally optically redirect light from theastigmatized refracted image to the screen intersection locations withoptimal pixel density. In this manner, the optical device optimizes theutilization of the projector's pixels by satisfying the image resolutionfor the portion of the field of view with a minimum of projector pixels.

Method 600 may include interposing a collimating mirror between the DEPand the curved screen in operation 616. Accordingly, a viewer at the DEPis viewing the curved screen through the collimating mirror. In oneembodiment the method further comprises operation 620 which includesdetermining a compromise DEP for each of the plurality of portions ofthe field of view when there are two or more observers.

In one embodiment, method 600 includes adjusting (in operation 622) therefractive element by altering its optical effect. Optionally, therefractive element is adjusted to a freeform shape.

A user of the present method and system may choose any of the aboveimplementations, or an equivalent thereof, depending upon the desiredapplication. In this regard, it is recognized that various forms of thesubject software analysis method and system for determination offree-form could be utilized without departing from the scope of thepresent technology and various implementations as disclosed.

As is evident from the foregoing description, certain aspects of thepresent implementation are not limited by the particular details of theexamples illustrated herein, and it is therefore contemplated that othermodifications and applications, or equivalents thereof, will occur tothose skilled in the art. It is accordingly intended that the claimsshall cover all such modifications and applications that do not departfrom the scope of the present implementation(s). Accordingly, thespecification and drawings are to be regarded in an illustrative ratherthan a restrictive sense.

Certain systems, apparatus, applications or processes are describedherein as including a number of modules. A module may be a unit ofdistinct functionality that may be presented in software, hardware, orcombinations thereof. When the functionality of an analysis module isperformed in any part through software, the module includes acomputer-readable medium. The analysis modules may be regarded as beingcommunicatively coupled. The inventive subject matter may be representedin a variety of different implementations of which there are manypossible permutations.

The methods described herein do not have to be executed in the orderdescribed, or in any particular order. Moreover, various activitiesdescribed with respect to the methods identified herein can be executedin serial or parallel fashion. In the foregoing Detailed Description, itcan be seen that various features are grouped together in a singleembodiment for the purpose of streamlining the disclosure. This methodof disclosure is not to be interpreted as reflecting an intention thatthe claimed embodiments require more features than are expressly recitedin each claim. Rather, as the following claims reflect, inventivesubject matter may lie in less than all features of a single disclosedembodiment. Thus, the following claims are hereby incorporated into theDetailed Description, with each claim standing on its own as a separateembodiment.

In an example implementation, the machine operates as a standalonedevice or may be connected (e.g., networked) to other machines. In anetworked deployment, the machine may operate in the capacity of aserver or a client machine in server-client network environment, or as apeer machine in a peer-to-peer (or distributed) network environment. Themachine may be a server computer, a client computer, a personal computer(PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant(PDA), a cellular telephone, a web appliance, a network router, switchor bridge, or any machine capable of executing a set of instructions(sequential or otherwise) that specify actions to be taken by thatmachine or computing device. Further, while only a single machine isillustrated, the term “machine” shall also be taken to include anycollection of machines that individually or jointly execute a set (ormultiple sets) of instructions to perform any one or more of themethodologies discussed herein.

The example computer system and client computers can include a processor(e.g., a central processing unit (CPU) a graphics processing unit (GPU)or both), a main memory and a static memory, which communicate with eachother via a bus. The computer system may further include avideo/graphical display unit (e.g., a liquid crystal display (LCD) or acathode ray tube (CRT)). The computer system and client computingdevices can also include an alphanumeric input device (e.g., akeyboard), a cursor control device (e.g., a mouse), a drive unit, asignal generation device (e.g., a speaker) and a network interfacedevice.

The drive unit includes a computer-readable medium on which is storedone or more sets of instructions (e.g., software) embodying any one ormore of the methodologies or systems described herein. The software mayalso reside, completely or at least partially, within the main memoryand/or within the processor during execution thereof by the computersystem, the main memory and the processor also constitutingcomputer-readable media. The software may further be transmitted orreceived over a network via the network interface device.

The term “computer-readable medium” should be taken to include a singlemedium or multiple media (e.g., a centralized or distributed database,and/or associated caches and servers) that store the one or more sets ofinstructions. The term “computer-readable medium” shall also be taken toinclude any medium that is capable of storing or encoding a set ofinstructions for execution by the machine and that cause the machine toperform any one or more of the methodologies of the presentimplementation. The term “computer-readable medium” shall accordingly betaken to include, but not be limited to, solid-state memories, andoptical media, and magnetic media.

The various optical remapping configurations and implementations shownabove illustrate remapping projected pixel locations with minimalaberration. A user of the present technology as disclosed may choose anyof the above implementations, or an equivalent thereof depending uponthe desired application. In this regard, it is recognized that variousforms of the subject optical remapping method and apparatus could beutilized without departing from the scope of the present invention.

As is evident from the foregoing description, certain aspects of thepresent technology as disclosed are not limited by the particulardetails of the examples illustrated herein, and it is thereforecontemplated that other modifications and applications, or equivalentsthereof, will occur to those skilled in the art. It is accordinglyintended that the claims shall cover all such modifications andapplications that do not depart from the scope of the present technologyas disclosed and claimed.

Other aspects, objects and advantages of the present technology asdisclosed can be obtained from a study of the drawings, the disclosureand the appended claims.

What is claimed is:
 1. A method for optically remapping pixels of animage onto a display screen of a simulator, comprising: projecting afrustum of light comprising the image from an origin point of aprojector of the simulator along an optical path to a design eye pointof the simulator, the frustum being rectangular and comprising: a firstaxis that is oriented approximately horizontally, the first axiscomprising a first magnitude; and a second axis that is orientedapproximately vertically, the second axis comprising a second magnitudethat is less than the first magnitude; refracting the image with arefractive lens positioned in the optical path between the projector andthe display screen, wherein a center of curvature of the refractive lensis approximately coincident with the origin point of the projector;reflecting the image with a reflective surface of a reflective elementpositioned in the optical path between the refractive lens and thedisplay screen, wherein the reflective surface is configured to provideanisotropic magnification of the image; and projecting the image onto adisplay surface of the display screen, wherein the display surface iscurved and is positioned in the optical path between the reflectiveelement and the design eye point, wherein a pixel density of the imageon the display surface is uniform, and wherein the display surfacecomprises: a first arc that is oriented approximately horizontally, thefirst arc having a first length; and a second arc that is orientedapproximately vertically, the second arc having a second length that isless than the first length.
 2. The method of claim 1, wherein theprojector is positioned to project the image onto a convex surface ofthe display screen.
 3. The method of claim 1, further comprisinginterposing a collimating mirror in the optical path between the designeye point and the display screen, wherein the image on the displaysurface is viewable in the collimating mirror from the design eye point.4. The method of claim 1, wherein a horizontal field of view of thedisplay screen exceeds 180 degrees, and wherein the display screen is asection of one of an ellipsoid, a toroid, and a minor variation from theellipsoid or toroid.
 5. The method of claim 1, wherein the refractivelens comprises a focal surface encompassing a location of a vertex ofthe frustum projected from the projector.
 6. The method of claim 1,wherein the refractive lens has an optical surface comprising one ormore of: a shape of a freeform; and a Gaussian curvature ofapproximately zero.
 7. The method of claim 1, further comprisingadjusting the refractive lens by altering its optical effect.
 8. Themethod of claim 1, further comprising adjusting an optical effect of thereflective element by changing the reflective surface to a freeformshape, wherein adjusting the optical effect of the reflective elementincludes one or more of mechanical deformation and thermal deformationof the reflective surface.
 9. The method of claim 1, further comprisingorienting the refractive lens such that a longest dimension of its focalsurface is substantially orthogonal to a longest dimension of thereflective surface of the reflective element.
 10. A display system,comprising: a projector to project a frustum of light comprising animage from an origin point of the projector along an optical path to adesign eye point of the display system, the frustum being rectangularand comprising: a first axis that is oriented approximatelyhorizontally, the first axis comprising a first magnitude; and a secondaxis that is oriented approximately vertically, the second axiscomprising a second magnitude that is less than the first magnitude; arefractive lens to refract the image, the refractive lens positioned inthe optical path between the projector and the design eye point, whereina center of curvature of the refractive lens is approximately coincidentwith the origin point of the projector; a reflective element with areflective surface to reflect the image, the reflective elementpositioned in the optical path between the refractive lens and thedesign eye point, wherein the reflective surface is configured toprovide anisotropic magnification of the image; and a display screenwith a display surface, wherein the display surface is curved and ispositioned in the optical path between the reflective element and thedesign eye point, wherein when the image is projected onto the displaysurface, a pixel density of the image is uniform, and wherein thedisplay surface comprises: a first arc that is oriented approximatelyhorizontally, the first arc having a first length; and a second arc thatis oriented approximately vertically, the second arc having a secondlength that is less than the first length.
 11. The display system ofclaim 10, wherein the projector is positioned to project the image ontoa convex surface of the display screen.
 12. The display system of claim10, further comprising a collimating mirror interposed in the opticalpath between the design eye point and the display screen, wherein theimage projected onto the display surface is viewable in the collimatingmirror from the design eye point.
 13. The display system of claim 10,wherein a horizontal field of view of the display screen exceeds 180degrees, and wherein the display screen is a section of one of anellipsoid, a toroid, and a minor variation from the ellipsoid or toroid.14. The display system of claim 10, wherein the refractive lenscomprises a focal surface encompassing a location of a vertex of thefrustum when it is projected by the projector.
 15. The display system ofclaim 10, wherein the refractive lens has an optical surface comprisingone or more of: a shape of a freeform; and a Gaussian curvature ofapproximately zero.
 16. The display system of claim 10, wherein anoptical effect of the refractive lens is adjustable.
 17. The displaysystem of claim 10, wherein an optical effect of the reflective elementis adjustable.
 18. The display system of claim 17, wherein the opticaleffect of the reflective element is adjustable by one or more ofmechanical deformation and thermal deformation of the reflective surfaceof the reflective element.
 19. The display system of claim 10, whereinthe reflective surface of the reflective element is adjustable to afreeform shape.
 20. The display system of claim 10, wherein a longestdimension of a focal surface of the refractive lens is orientedsubstantially orthogonal to a longest dimension of the reflectivesurface of the reflective element.